Goto

Collaborating Authors

 vision language model


#AIES2026 social media round-up

AIHub

This week saw researchers gather in Malmö at the ninth AAAI / ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) . We cast an eye over social media platforms to find out what participants got up to at the event. The 9th annual AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES-26) is underway in Malmö. We are looking forward to a robust program full of educational and networking opportunities. Today at #AIES2026 in Malmö I joined the panel on Participation, Sovereignty, and Power in AI Governance with The Mitigation Paradox, joint work with Mina Schütz, Martin Kappes and Marc-Oliver Pahl. 3 takeaways: https://doi.org/10.5281/zenodo.23302284


What's on the programme at #AIES2026?

AIHub

The ninth AAAI / ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) is taking place in Malmö, Sweden from 12-14 October 2026. The panel discussion will take place on Tuesday 13 October. Find out more at the conference webpage . Lucy Smith is Senior Managing Editor for AIhub. Lucy Smith is Senior Managing Editor for AIhub.


Generative AI has changed mathematics forever. Where to from here?

AIHub

Generative AI has changed mathematics forever. In August, artificial intelligence company OpenAI announced ten advances in mathematics and computer science made with their as-yet unreleased model Astra. The discoveries cover a wide range of mathematical fields, including geometry, cryptography and coding theory. These ten are just the latest in a rapidly growing list of mathematical breakthroughs made by generative artificial intelligence (AI) systems. OpenAI accompanied the announcement with a statement on "responsibility to the mathematical community".


Can you teach yourself to detect AI writing? Maybe

AIHub

Can you teach yourself to detect AI writing? AI is quietly reshaping writing. It's not just that prose is changing -- the very structure of written language is genuinely changing too. Did this article's introduction seem AI generated? What made you feel that way?


VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actions

Neural Information Processing Systems

Visual imitation learning (VIL) provides an efficient and intuitive strategy for robotic systems to acquire novel skills. Recent advancements in Vision Language Models (VLMs) have demonstrated remarkable performance in vision and language reasoning capabilities for VIL tasks. Despite the progress, current VIL methods naively employ VLMs to learn high-level plans from human videos, relying on pre-defined motion primitives for executing physical interactions, which remains a major bottleneck. In this work, we present VLMimic, a novel paradigm that harnesses VLMs to directly learn even fine-grained action levels, only given a limited number of human videos. Specifically, VLMimic first grounds object-centric movements from human videos, and learns skills using hierarchical constraint representations, facilitating the derivation of skills with fine-grained action levels from limited human videos. These skills are refined and updated through an iterative comparison strategy, enabling efficient adaptation to unseen environments. Our extensive experiments exhibit that our VLMimic, using only 5 human videos, yields significant improvements of over 27% and 21% in RLBench and real-world manipulation tasks, and surpasses baselines by more than 37% in long-horizon tasks. Code and videos are available on our anonymous homepage.


Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions

Neural Information Processing Systems

Recent advancements in large vision language models have demonstrated remarkable proficiency across a wide range of tasks. Yet, these models still struggle with understanding the nuances of human humor through juxtaposition, particularly when it involves nonlinear narratives that underpin many jokes and humor cues. This paper investigates this challenge by focusing on comics with contradictory narratives, where each comic consists of two panels that create a humorous contradiction. We introduce the YesBut benchmark, which comprises tasks of varying difficulty aimed at assessing AI's capabilities in recognizing and interpreting these comics, ranging from literal content comprehension to deep narrative reasoning. Through extensive experimentation and analysis of recent commercial or open-sourced large vision language models, we assess their capability to comprehend the complex interplay of the narrative humor inherent in these comics. Our results show that even the state-of-the-art models still struggle with this task. Our findings offer insights into the current limitations and potential improvements for AI in understanding human creative expressions.


VFM-VLM: Vision Foundation Model and Vision Language Model based Visual Comparison for 3D Pose Estimation

arXiv.org Artificial Intelligence

Abstract--Vision Foundation Models (VFMs) and Vision Language Models (VLMs) have revolutionized computer vision by providing rich semantic and geometric representations. This paper presents a comprehensive visual comparison between CLIP based and DINOv2 based approaches for 3D pose estimation in hand object grasping scenarios. We evaluate both models on the task of 6D object pose estimation and demonstrate their complementary strengths: CLIP excels in semantic understanding through language grounding, while DINOv2 provides superior dense geometric features. Through extensive experiments on benchmark datasets, we show that CLIP based methods achieve better semantic consistency, while DINOv2 based approaches demonstrate competitive performance with enhanced geometric precision. Our analysis provides insights for selecting appropriate vision models for robotic manipulation and grasping, picking applications.


The promising potential of vision language models for the generation of textual weather forecasts

arXiv.org Artificial Intelligence

Despite the promising capability of multimodal foundation models, their application to the generation of meteorological products and services remains nascent. To accelerate aspiration and adoption, we explore the novel use of a vision language model for writing the iconic Shipping Forecast text directly from video-encoded gridded weather data. These early results demonstrate promising scalable technological opportunities for enhancing production efficiency and service innovation within the weather enterprise and beyond.


Table Comprehension in Building Codes using Vision Language Models and Domain-Specific Fine-Tuning

arXiv.org Artificial Intelligence

Building codes contain critical information for ensuring safety, regulatory compliance, and informed decision-making in construction and engineering. Automated question answering systems over such codes enable quick and accurate access to specific regulatory clauses, improving efficiency and reducing errors. Retrieval-Augmented Generation (RAG) systems are essential for this task as they combine the precision of information retrieval with the generative capabilities of language models. However, tabular data are challenging to extract as they often involve complex layouts, merged cells, multi-row headers, and embedded semantic relationships that are not easily captured by traditional natural language processing techniques and Vision Language Models (VLMs). This paper explores and compares two methods for extracting information from tabular data in building codes using several pre-trained VLMs. First, a direct input method is used, where the image of the page is input directly into the VLMs, which are then tasked with answering questions based on the image. Second, an indirect input method is introduced, which involves converting an image of a page containing tables into the LaTeX code and then answering inquires based on the LaTeX-based input. The experiments find that the direct input method generally resulted in higher accuracy than the indirect input method. To further improve the performance, we fine-tuned each VLM using Low Rank Adaptation (LoRA) on a domain-specific tabular dataset. The fine-tuned models exhibited substantial improvements, with Qwen2.5-VL-3B-Instruct achieving relative accuracy gains exceeding 100%. Our results highlight the potential of parameter-efficient fine-tuning methods to adapt powerful VLMs for understanding complex structured data in specialized fields, such as building code interpretation and regulatory compliance.


KV-Efficient VLA: A Method to Speed up Vision Language Models with RNN-Gated Chunked KV Cache

arXiv.org Artificial Intelligence

Vision-Language-Action (VLA) models offer a unified framework for robotic perception and control, but their ability to scale to real-world, long-horizon tasks is limited by the high computational cost of attention and the large memory required for storing key-value (KV) pairs during inference, particularly when retaining historical image tokens as context. Recent methods have focused on scaling backbone architectures to improve generalization, with less emphasis on addressing inference inefficiencies essential for real-time use. In this work, we present KV-Efficient VLA, a model-agnostic memory compression approach designed to address these limitations by introducing a lightweight mechanism to selectively retain high-utility context. Our method partitions the KV cache into fixed-size chunks and employs a recurrent gating module to summarize and filter the historical context according to learned utility scores. This design aims to preserve recent fine-grained detail while aggressively pruning stale, low-relevance memory. Based on experiments, our approach can yield an average of 24.6% FLOPs savings, 1.34x inference speedup, and 1.87x reduction in KV memory. Our method integrates seamlessly into recent VLA stacks, enabling scalable inference without modifying downstream control logic.